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Abstract 

This  paper  presents  experimental  results  for  the  Northrop  Grumman  resolution 
enhancement  and  non-uniformity  correction  algorithms.  The  results  include 
simulation  results  using  complex  images,  laboratory  measurements  of 
Modulation  Transfer  Function,  Minimum  Resolvable  Temperature  and  Noise 
Equivalent  Temperature  Difference,  and  results  obtained  using  microscanned 
flight  data  taken  with  a  midwave  IR  staring  sensor.  The  flight  data  includes  urban 
and  rural  environments.  These  results  show  that  resolution  enhancement  of 
greater  than  2:1  has  been  achieved  and  is  only  limited  by  the  optical  resolution  of 
the  sensor.  Non-uniformity  correction  without  the  use  of  calibration  data  is  shown 
to  reduce  the  fixed  pattern  offset  noise  to  less  than  the  temporal  noise,  even  after 
temporal  integration  of  multiple  frames  of  data.  Analytic  models  for  the  MTF  are 
presented  using  two  different  assumptions  regarding  resolution  enhancement. 
Comparison  with  the  laboratory  results  shows  that  the  effective  detector  size  is 
reduced  in  addition  to  the  expected  effects  of  increase  in  sampling  density. 

Introduction 


Microscanning  has  long  been  recognized  as  a  technique  for  increasing  the  effective 
resolution  of  staring  sensors.  Prior  work  can  be  broadly  classified  into  efforts  that 
only  considered  the  effect  of  increasing  the  sampling  density  to  reduce  aliasing 
caused  by  insufficient  sampling  of  the  spatial  frequencies  present  at  the  image 
plane  of  the  sensor,  and  efforts  that  attempted  to  recover  the  aliased  spatial 
frequencies  by  combining  the  data  from  multiple  frames  of  microscanned  imagery. 
While  both  methods  are  technically  feasible,  several  practical  problems  have 
restricted  the  use  of  these  techniques.  The  primary  problem  is  the  computation 
costs  when  the  scenes  being  microscanned  are  moving  in  an  irregular  fashion. 

Several  research  efforts  have  developed  sensors  that  provide  an  implicit 
computation  using  interlaced  images  from  consecutive  frames  to  achieve  an 
“apparent”  resolution  enhancement  of  typically  2:1 .  These  efforts  usually  take 
advantage  of  interlaced  video  displays  coupled  with  eye-brain  integration  to  provide 
increased  Modulation  Transfer  Function  (MTF)  within  the  limits  of  detector  aliasing. 
Aliasing  caused  by  the  finite  detector  aperture  limits  this  method,  even  under  static 
laboratory  conditions.  In  a  flight  environment,  the  need  for  a  known  amount  of 
interlace  is  very  difficult  to  achieve.  It  can  be  argued  that  if  the  field  of  view  of  the 
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sensor  is  small,  and  if  the  sensor  is  operated  so  as  to  stabilize  the  image  on  the 
focal  plane,  then  the  desired  enhancement  can  be  achieved.  For  the  evolving 
distributed  aperture  imaging  systems  which  have  been  our  focus  for  several  years^’ 
2, 3, 4, 5, 6^  neither  of  these  conditions  obtain. 

The  second  approach  which  we  and  others  have  been  pursuing  is  to  combine 
series  of  images  to  obtain  both  increased  sampling  density,  and  to  separate  the 
aliased  frequency  components,  using  them  to  reconstruct  the  higher  spatial 
frequencies  that  are  present  at  the  image  plane.  The  technical  feasibility  of  this 
was  shown  using  a  Maximum  Likelihood  Estimator  (MLE)  algorithm^’  Their 
well  known  enhancement  of  static  IR  data  is  shown  in  Figure  1 . 


Figure  1  -  Resolution  Enhancement  Using  Maximum  Likelihood  Method 

The  images  are  taken  with  a  1 28x1 28  MWIR  staring  sensor.  A  series  of  1 6  images 
was  processed  to  achieve  a  4:1  resolution  enhancement  shown  in  the  right  hand 
Image.  Although  the  scene  was  static,  this  fact  is  not  essential  to  the  enhancement 
algorithm  which  they  used.  In  fact,  the  sampling  grid  was  not  uniform  due  to  an 
experimental  problem,  but  the  output  image  still  contained  frequency  components 
beyond  the  detector  limit  of  one  cycle  per  detector  aperture.  The  primary  problem 
with  this  algorithm  was  the  computational  burden  which  could  be  as  high  as 
thousands  of  arithmetic  operations  per  input  sample.  However,  the  strengths  of  the 
algorithm  are  very  significant.  There  is  no  a  priori  assumption  about  the  nature  of 
the  image,  except  that  it  is  temporally  invariant  over  the  duration  of  the  microscan 
sequence,  and  that  it  has  finite  energy.  The  excellent  work  by  this  group  has  been  a 
major  reason  that  we  have  continued  to  pursue  resolution  enhancement  for  the 
distributed  aperture  sensor  system. 


In  this  paper,  the  results  of  applying  alternate  algorithms  that  have  some  similarities 
to  the  Maximum  Likelihood  method  will  be  presented.  A  major  focus  of  this  work  is 
the  reduction  in  computation  costs.  The  very  high  payoff  for  increased  resolution  of 
distributed  aperture  sensors  can  be  achieved  if  the  costs  for  processing  are 
containable  within  the  resources  of  current  and  planned  military  aircraft.  We  have 
succeeded  in  reducing  computation  costs  below  50  operations  per  input  sample 
while  achieving  2X  resolution  enhancements  that  are  similar  to  the  results  from  the 
MLE  algorithm.  This  improves  on  the  results  reported  in  reference  1 1  by  about  4:1 . 
Of  equal  importance,  the  algorithms  are  shown  to  handle  flight  data  and  provide  a 
degree  of  scene-based  non-uniformity  correction  (NUC)  not  previously  achieved. 
The  paper  describes  four  systems  (two  simulations  and  two  experiments  using 
actual  sensor  data)  and  compares  the  results  with  models  that  define  the  projected 
performance  of  the  sensors  when  resolution  enhancement  and  NUC  are  provided. 
The  performance  is  compared  with  the  performance  predictions  for  an  oversampled 
sensor  as  well  as  one  in  which  the  detector  sizes  are  reduced  to  match  the 
increased  sampling  density. 

Resolution  Enhancement  Model 


Figure  2  -  Estimating  High  Resolution  Images 

The  general  form  of  the  resolution  enhancement  model  is  show  in  Figure  2.  Scene 
data  is  modulated  by  the  optics  before  arriving  at  the  image  plane  as  a  two- 
dimensional  image.  The  staring  detector  system  samples  this  image  using  both  a 
spatial  sampling  grid  and  the  detector  aperture  function.  Microscanning  provides  a 
series  of  sampled  data  sets  at  varying  sample  phases.  If  the  scene  is  treated  as 
a  time  invariant  function,  then  the  multiple  data  sets  can  be  combined  to  obtain  a 
single  data  set  that  has  been  sampled  at  higher  sampling  density.  MLE  takes  this 
data  set,  and  compares  it  with  a  set  that  is  generated  by  the  algorithm,  then 
sampled  in  the  same  way  that  the  detectors  sampled  the  original  image.  The 
difference  between  the  two  data  sets  is  used  to  refine  the  estimates  of  the  individual 
data  values  using  a  recursive  steepest  descent  method.  In  general,  any  component 
of  the  input  image  that  modifies  the  detector  data  set  when  sampling  phase  is 
varied  can  be  recovered  in  the  output  data  set.  Since  the  input  data  set  is  limited 
by  the  optics,  the  recoverable  data  is  similarly  limited.  The  sampling  limits  of  the 


detectors  I  have  called  the  Nyquist  Limit,  while  the  optics  limits  I  have  designated 
the  Rayleigh  Limit. 


Notice  that  we  could  close  the  estimation  loop  around  an  optics  MTF  which  is 
equivalent  to  de-convolving  the  optics.  However,  spatial  frequencies  beyond  the 
optics  cut-off  do  not  affect  the  scene  presented  to  the  detectors,  and  therefore 
cannot  be  recovered  by  this  method.  The  usual  techniques  described  in  the 
literature  for  exceeding  the  Rayleigh  Limit  involve  some  statistical  Inference  or  other 
method  of  describing  the  necessary  higher  frequencies.  One  of  the  more 
successful  methods  has  been  the  resolution  of  stellar  binary  objects  where  the 
existence  of  a  binary  pair  is  assumed  in  the  enhancement  process. 
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Figure  3  -  Effect  of  Detector  Resolution  Enhancement  on  MTF 

Figure  3  illustrates  the  trades  for  different  algorithm  choices.  The  Optics  MTF  for  a 
Gaussian  blur  spot  is  the  best  available  resolution  for  algorithms  that  operate  only 
on  detector  data.  A  typical  optics  f#  of  2.3  with  diffraction-limited  optics  was 
assumed.  The  optical  spatial  frequencies  extend  to  about  1 1 0  cycles  per 
millimeter.  To  provide  a  Nyquist  Frequency  that  exceeds  the  Rayleigh  Limit 
requires  detector  sample  spacing  of  4.5  micrometers.  This  is  much  smaller  than  can 
be  provided  with  current  detector  technology. 

If  the  effect  of  a  selected  algorithm  is  to  increase  sample  density  without  reducing 
the  effective  detector  aperture,  then  the  best  available  MTF  is  shown  by  the  curve 
labeled  Oversample  Only.  When  the  effect  of  the  algorithm  is  to  reduce  the 
effective  aperture  size,  then  the  curves  labeled  Samples/Dwell  =  1,2,  and  3  apply. 
The  Nyquist  Frequency  is  increased  as  sample  density  increases,  and  detector 
limits  are  also  Increased.  It  Is  this  effective  reduction  in  detector  aperture  that  is 


the  primary  attraction  of  the  more  complex  MLE  algorithms.  The  cases  illustrated 
are  for  a  25  micrometer  detector  at  a  nominal  wavelength  of  4  micrometers. 

The  experimental  data  that  we  have  used  was  taken  with  arrays  having  detector 
spacing  of  38  and  50  micrometers  with  operation  in  the  MWIR  band.  For  these  two 
cases,  the  available  resolution  enhancement  using  MLE  is  8.4:1  and  1 1 :1 
respectively.  With  conventional  high  resolution  MWIR  focal  planes  having  20-25 
micrometer  detectors,  the  available  resolution  enhancement  is  about  4.5-6:1  which 
is  still  a  substantial  increase. 

Algorithm  Simulation  Results 

An  algorithm  simulation  software  platform  was  constructed  fortesting  algorithms. 
This  can  accept  either  synthetic  scenes  or  real  images.  To  answer  questions  of 
enhancement  accuracy,  it  typically  uses  a  high  resolution  scene  that  is  “scanned”  by 
a  simulated  detector  array  to  achieve  a  lower  resolution.  Intermediate  resolutions 
are  also  available  that  match  the  degree  of  resolution  enhancement  that  is  being 
implemented.  This  permits  direct  comparison  between  the  resolution  achieved  by 
an  algorithm  and  the  resolution  that  is  provided  by  an  “ideal”  detector  with  the 
enhanced  resolution.  The  simulation  includes  microscanning  patterns  and  the 
ability  to  move  the  line  of  sight  of  the  sensor  in  varying  step  sizes  across  the  image. 
Addition  of  both  fixed  pattern  noise  and  temporal  noise  is  provided  to  permit 
evaluation  of  the  ability  to  do  NUC  and  to  determine  the  ratio  of  output  noise  to  input 
noise  for  both  modes.  A  variety  of  scenes  have  been  obtained  by  digitizing  visible 
wavelength  photographs.  A  typical  source  scene  provides  1 400x2000  pixels. 
Unenhanced  detector  array  size  is  64x64  elements  with  each  detector  covering  a 
4x4  array  of  source  samples.  Motion  steps  up  to  5  detector  elements  per  frame 
with  a  step  size  resolution  of  y4  detector  element  are  provided.  The  platform  runs 
under  Windows  95.  Using  a  1 00  MHz  Pentium  laptop  computer,  the  complete 
code,  including  detector  simulation,  resolution  enhancement  and  display  of  256x256 
result  images  operates  at  about  1/2  frame  per  second.  On  a  laboratory  350  MHz 
PC  with  better  graphics,  rates  of  2.5  frames  per  second  are  obtained.  There  was 
no  attempt  to  optimize  this  code  for  speed. 


Unenhanced  Image  2:1  Enhanced  Image 

Figure  4 -2:1  Resolution  Enhancement  Simuiation 

Figure  4  shows  the  source  image  at  the  detector  resoiution  and  the  resuit  of 
applying  the  enhancement  algorithms  using  a  2:1  Microscan  pattern.  The  source 
seiected  for  this  image  is  the  Abbey  L’homme  in  Caen,  Normandy.  It  is  used 
because  it  has  a  broad  mixture  of  image  patterns  including  repetitive  lattices, 
smooth  curves  and  a  mixture  of  diagonai  elements.  In  Figure  5,  the  enhancement 
of  the  detector  image  at  a  2:1  enhancement  ratio  is  compared  with  the  ideai  image 
that  results  when  the  source  is  sampied  using  a  detector  size  that  is  1/2  of  the 
unenhanced  detector  size.  The  difference  between  these  two  images  is  not 
perceptibie,  but  measurements  show  that  the  error  in  this  noise-free  case  is  about 
two  percent  of  the  signai. 


Unenhanced  Image  +  FPN  Enhanced  Image 

Figure  6  -  Enhancement  Algorithm  Eliminates  Fixed  Pattern  Noise 


Figure  6  shows  the  addition  of  a  large  amount  of  fixed  pattern  noise  to  the  detector 
image.  The  enhanced  image  has  no  discernible  fixed  pattern  noise,  due  to  the  use 
of  image  gradients  as  a  part  of  the  enhancement  algorithm  structure.  This  is 
verified  by  the  experimental  results  discussed  below. 


R  '  IS 

p . 

pi 

Unenhanced  Image  +  Temporal  Noise 


Enhanced  Image 


Figure  7  -  Temporal  Noise  is  Unchanged  by  Enhancement  Algorithm 

Figure  7  shows  the  addition  of  temporal  noise  only  to  the  image,  and  the  resulting 
enhanced  image.  Here  the  noise  is  visible  on  the  output.  The  figure  of  merit  is  the 
ratio  of  sample  noise  on  the  output  to  sample  noise  on  the  input.  For  the  particular 
form  of  the  algorithms  demonstrated  here,  the  theoretical  ratio  is  1 .06.  The 
measured  ratio  is  typically  within  ten  percent  of  this  value. 

Since  it  is  difficult  to  separate  the  visual  content  from  the  algorithm  performance, 
this  simulation  was  modified  to  show  a  standard  4-Bar  target  resolution 
enhancement.  A  specific  point  of  interest  is  to  show  a  case  where  the  spatial 
frequency  of  the  bars  exactly  matches  the  detector  response  null  (one  cycle  per 
detector). 

It  is  usual  to  refer  to  such  targets  as  having  a  single  spatial  frequency.  However,  the 
spatial  frequencies  present  in  this  target  span  a  broad  range  of  frequencies 
because  of  the  portions  that  do  not  have  the  bars.  Figure  8  shows  the  difference  in 
spectra  for  a  four  bar  target  and  for  a  repetitive  square  wave  pattern  extending  over 
the  entire  field  of  view.  The  4-Bar  pattern  that  has  bars  and  spaces  that  are  two 
units  wide  for  a  total  4-Bar  width  of  1 4  units  in  a  data  series  of  1 28  units.  As 
expected  the  repetitive  pattern  has  all  of  its  energy  concentrated  at  a  single 
frequency.  The  4-Bar  target  has  its  energy  spread  over  the  entire  range  of 
frequencies  with  relatively  little  at  the  4-Bar  “frequency”.  The  difference  between 
these  patterns  determines  the  degree  to  which  the  4-Bar  series  is  aperiodic.  A 
completely  aperiodic  series  will  exhibit  a  spectrum  with  uniform  amplitude  at  each 
frequency.  The  reconstruction  of  the  4-Bar  pattern  is  only  in  error  by  the  amount  that 


signals  near  the  4-Bar  frequency  are  lost.  As  noted  above,  for  typical  scenes,  this 
is  a  small  percentage  error  in  reconstructing  the  high-resolution  image. 


Frequency  -  Cycles/Sample 

Figure  8  -  Comparison  of  Singie  Frequency  and  4-Bar  Spectra 


Figure  9  shows  the  source  image  for  the  4-Bar  pattern.  The  simulated  detector  has 
a  span  of  one  4-Bar  cycle  for  the  target  below  the  right  edge  of  the  large  target. 
Figure  10  shows  the  simulated  unenhanced  detector  data  compared  with  an  ideal 
detector  that  is  y2  the  size  in  both  dimensions.  Figures  1 1  and  1 2  show  the  result  of 
the  enhancement  algorithms  for  the  best  and  worst  sampling  phase  between  the 
detectors  and  the  image.  When  motion  is  added  to  microscanning,  the  effects  of 
sampling  phase  can  be  discerned  as  a  modulation  of  the  amplitude  for  the  target 
that  is  at  the  enhanced  Nyquist  Limit.  This  cannot  readily  be  portrayed  in  static 
images  but  will  be  illustrated  dynamically  during  the  presentation.  The  key  result 
from  this  simulation  is  the  ability  to  derive  signals  at  the  detector  null  provided  that 
they  are  immersed  in  an  aperiodic  sequence.  Flowever,  it  is  also  notable  that 
signals  having  spatial  frequency  content  beyond  the  detector  null  are  also 
recovered,  including  signals  at  the  second  detector  null.  Of  course,  in  this 
simulation  there  is  no  Rayleigh  Limit  to  contend  with  as  the  high  resolution  signals 
were  generated  analytically. 


Figure  9  -  Simulated  4-Bar  Targets  with  Periods  Matched  to  Detector  Size 


Unenhanced  Image  Ideal  Ground  Truth  Image 

Figure  10  -  Unenhanced  4-Bar  Images  vs.  Ideal  2X  Enhanced  Images 


Enhanced  Image 


Ideal  Ground  Truth  Image 


Figure  11  -  Enhanced  4-Bar  Images  at  Poorest  Sampling  Phase 

(Aiias  Canceiing) 


Enhanced  Image  Ideal  Ground  Truth  Image 

Figure  12  -  Enhanced  4-Bar  Images  at  Best  Sampling  Phase  (Aiias  Adding) 


Laboratory  Experimental  Results 
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Figure  13-  DAIRS  Laboratory  Algorithm  Verification  Equipment 

The  algorithms  have  been  transferred  to  the  Distributed  Aperture  IR  Sensor 
(DAIRS)  laboratory  (Figure  13).  The  sensor  for  these  experiments  is  the  Amber 
Radiance  I  camera  with  256x256  detectors.  The  laboratory  tests  have  included 
measurements  of  MTF,  MRTD  and  NUC,  plus  error  measurements  of  scene-based 
motion  measurement  algorithms.  MTF  was  determined  using  the  digital  data  such 
as  that  shown  in  Figure  14.  Selected  rows  were  analyzed  using  Mathcad  to 
determine  modulation.  Maximum,  minimum  and  average  excursions  were 
determined  for  multiple  Instances. 
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Figure  14  -  Laboratory  Tests  Verify  Resolution  Enhancement 


The  2X  enhancement  cases  (Half  Pixel  microscan)  were  analyzed  using  single 
horizontal  data  lines  and  are  plotted  in  Figure  1 5  which  also  compares  these  results 
with  MTF  models  using  two  different  assumptions.  The  Interlaced  Model  assumes 
that  the  sample  density  is  increased  by  2X  while  the  detector  aperture  is 
unchanged.  The  Maximum  Likelihood  (MLE)  Model  assumes  that  the  sample 
density  is  increased  by  2X  and  that  the  detector  aperture  is  halved.  The  lowest 
curve  shows  the  unenhanced  prediction,  the  middle  curve  shows  the  Interlaced 
Model  prediction,  and  the  upper  curve  shows  the  MLE  Model  prediction.  Actual 
data  is  between  the  upper  and  middle  curves,  but  extends  to  frequencies  well 
beyond  Nyquist  as  is  expected  from  the  composite  image  shown  in  Figure  14.  It  is 
clear  that  the  measured  MTF  routinely  exceeds  the  predicted  MTF  for  the  Interlaced 
Model.  The  observed  deficit  relative  to  the  MLE  Model  prediction  has  not  been 
explained,  but  may  be  due  to  non-ideal  detectors  or  optics. 
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Figure  15  -  Comparison  of  Experimental  MTF  with  Two  Models 

Measurement  of  MRTD  uses  a  conventional  arrangement  of  4-Bar  targets  with  a 
calibrated  source.  Temperature  difference  is  controlled  to  obtain  transitions  from 
visible  to  not  visible  and  vice  versa  for  both  colder  and  hotter  than  background 
transitions.  Multiple  observers  and  multiple  sequences  with  each  observer  were 


used.  To  provide  the  30  Hz  frame  rate  needed  for  correct  eye  integration,  the  size 
of  each  frame  has  been  reduced  to  allow  use  of  a  standard  Sun  work  station  as  the 
driver.  Figure  1 6  shows  the  MRTD  results  for  one  observer  with  unenhanced  and 
2:1  resolution  enhancement  images. 
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Figure  16 -Measured  MRTD 

As  a  part  of  the  diagnosis  of  the  MTF  deficit,  measurements  of  the  ability  to 
determine  microscan  motion  from  scene  data  were  repeated  in  the  laboratory.  The 
error  between  measured  displacement  and  scene  based  estimated  displacement 
has  a  standard  deviation  of  less  than  0.01  detector  IFOV,  but  does  exhibit  a  scale 
factor  bias  of  about  5%  which  is  being  investigated.  The  measurement  method  did 
not  provide  information  concerning  absolute  registration.  For  resolution 
enhancement  algorithms  of  the  MLE  type,  the  short  term  repeatability  is  the  main 
error  of  concern.  Methods  from  the  open  literature^^,  suggest  poorer  results  than 
obtained  here,  but  those  methods  do  not  include  corrections  for  errors  caused  by 
aliasing  and  fixed  pattern  noise. 

The  algorithms  use  the  detector  difference  operator  for  NUC  which  has  been 
presented  in  prior  papers'^’  For  convenience,  the  equations  are  repeated  here. 

If  we  have  two  sets  of  data  from  detectors  with  the  same  offsets,  then  we  can 
eliminate  the  offset  by  subtracting  one  set  from  the  other  set.  If  the  detectors  are 
looking  at  the  same  scene  location,  we  will  also  remove  the  scene  signal.  When 


microscanning,  or  otherwise  moving  the  line  of  sight,  the  detector  differences  will 
contain  scene  differences  without  offsets.  For  any  detector: 


D-f  =  S-f  +  OffsBt-f 

D2  =  S2  +  0ffS&t2 

But  Offseti  =  Offset2  =  Offset 

D2  ■  =  S2  ~  Si  +  Offset  -  Offset  =  S2  ~  Si 


Notice  that  there  is  no  need  to  estimate  the  offset  or  perform  any  added  arithmetic 
to  remove  it.  The  resulting  signals  are  spatial  differences  which  are  used  to 
reconstruct  the  intensity  image.  The  primary  draw  back  of  this  approach  is  the 
need  to  know  the  positions  of  the  two  signals  for  purposes  of  reconstruction.  In  the 
case  of  resolution  enhancement,  such  knowledge  is  essential  for  MLE  class 
algorithms  and  can  be  derived  using  the  MLE  approach. 

A  definitive  test  for  NUC  is  to  perform  frame  integration  over  a  large  number  of 
frames,  and  to  plot  the  total  noise  versus  number  of  frames.  When  the  noise  is 
purely  temporal  (uncorrelated  from  frame  to  frame),  the  total  noise  will  increase 
proportional  to  the  square  root  of  the  number  of  frames  integrated,  while  the 
sensitivity  will  Increase  proportional  to  the  number  of  frames.  Thus,  the  NETD  will 
decrease  proportional  to  the  inverse  of  the  square  root  of  the  number  of  frames  as 
shown  in  Figure  1 7.  The  experimental  data  shown  In  the  figure  indicates  that  the 
fixed  pattern  noise  is  reduced  to  less  than  2-3  percent  of  temporal  noise  and  can  be 
interpreted  as  validating  reduction  of  fixed  pattern  noise  to  zero  as  predicted  by  the 
equations  above. 
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Figure  17  -  Frame  Integration  Test  Confirms  NUC  Prediction 


Flight  Data  Analysis 


A  prior  internally-funded  program  flew  a  prototype  sensor  with  a  128x128  InSb 
array.  The  sensor  used  a  25  mm  lens  with  f#  2.3.  Detector  size  was  50x50 
micrometers  nominal.  Sensor  data  was  collected  as  1 2  bit  samples  at  a  frame  rate 
of  60  Hz.  Microscan  motion  was  provided  by  a  mirror  driven  by  a  commercial 
piezo-electric  actuator.  Typical  frame  integration  time  was  about  one  millisecond. 
Cooling  was  provided  by  an  LN2  pour  dewar  which  was  vented  to  the  atmosphere. 
The  sensor  was  flown  on  a  BAG  1  -1 1  used  at  our  Baltimore  facility  for  a  variety  of 
experimental  programs.  Calibration  was  performed  immediately  prior  to  each  flight 
using  external  equipment.  Variations  in  LN2  temperature  were  caused  by  altitude 
changes.  These,  in  turn  caused  large  variations  from  the  calibrated  fixed  pattern 
noise. 

The  algorithms  used  in  the  laboratory  have  been  adapted  for  analysis  of  this  flight 
data.  The  primary  changes  are  the  need  to  incorporate  an  optical  flow  model  that 
allows  image  motion  to  vary  over  the  field  of  view.  This  change  adds  a  significant 
computational  burden,  but  has  been  successfully  demonstrated  to  recover  flight 
scene  motions  within  about  .025  source  samples  (1  a).  A  variety  of  error  sources 
have  been  identified  and  corrected  to  achieve  this  result.  The  motion  history  for  a 
short  segment  of  flight  data  is  shown  in  Figure  1 8.  The  nominal  microscan  motion 
is  a  square  measuring  one-half  source  pixel  in  each  direction.  There  was  no  in 
flight  measurement  due  to  the  nature  of  the  equipment  used  to  drive  the 
microscanner.  The  commanded  position  was  recorded. 


Figure  18  -  Flight  Data  Motion  Estimates 


Figure  19  -  Flight  Data  Showing  Resoiution  Enhancement  and  NUC 

Figure  19  illustrates  the  resolution  enhancement  and  NUC  achieved  with  the  flight 
data.  The  non-uniformity  is  fully  corrected  without  use  of  any  offset  calibration  table. 
The  white  spot  at  the  lower  right  edge  is  caused  by  a  group  of  bad  detectors  that 
flashed  irregularly  from  zero  to  full  scale.  To  assess  the  resolution  enhancement 
separate  from  the  non-uniformity  correction,  the  enhanced  image  has  been 
resampled  to  the  detector  resolution.  Figure  20  compares  the  resampled  image 
with  the  enhanced  image.  The  loss  of  detail  in  the  original  image  was  clearly 
impacted  by  the  large  fixed  pattern  noise.  There  is  still  a  significant  improvement  in 
image  quality. 


Figure  20  -  Comparison  of  Enhanced  Image  with  Simulated  Source 


Conclusions 


Algorithms  have  been  developed  that  provide  Resolution  Enhancement,  Non¬ 
uniformity  Correction  and  Sensitivity  Enhancement  using  microscanned  IR  images. 
The  algorithms  differ  from  prior  efforts  primarily  in  speed.  This  speed  improvement 
provides  a  path  to  a  real  time  application  of  these  techniques  which  will  significantly 
improve  the  performance  of  imaging  sensors.  The  performance  of  the  algorithms 
has  been  verified  using  simulated  data  and  real  sensor  data  from  both  laboratory 
and  flight.  Laboratory  tests  have  confirmed  the  analytic  models  used  to  predict  the 
performance  of  these  enhanced  images. 
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